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Abstract- . ' 

> - , • > 

The purpose of the study was to. examine test bias 'and the "non-effects" 

of schooling. Teachers were given a list of words selected from standardized 

vocabulary tests and asked to indicate the wor'ds-they had taught. The wdrds 

> »• 

were classified by the grade level at which they were first introduced. 
Ninety-five third grade students in four schools were given two seventeen 
item subtests: subtests of. items the students did and .did not have an 
opportunity to learn (Test Y and Test N, respectively) . The difference ' 

in the scores on Test Y and Test N was used as an index of school effects 

. *•.,.. v 

on student vocabulary. Achievement test bias (when equated' with content 
validity) was present. Students idored higher on the subtest containing ' 

\ f 

items they had had an opportunity to learn (p<.01). * 



Opportunity 'to' Learn, Test Bias, and School Effects 

' Introduction : 

Two conclusions which seem to have been largely accepted by educators' 
in recent years concern the test bias of various standardized tests (Thorndike, 

1971) and the apparent non-effects of schooling (Coleman, 1966), The 

<* * 

acceptance of research findings concerning the bias of tests -for students 

* * 

living iR minority subcultures was* largely responsible for the attempted \ 

movement away from the use of tests altogether or, at the very. least, the 

movement: toward "culture faif*". tes-ts. * The- acceptance of the findings con- ' 

cerning the impotence of^jschools in producing student learning was in 

• ft . 
part responsible f or % the anti-*£hoo1 on de-scho.oling movement. This paper 

is an attempt to examine tire ffyys'tlens'of test bias and non-effects of , 

schooling in a different light. 

Recently, Lewy (1972) investigated the importance of a variable termed 

"opportunity to learn" in relationship to student achievement. Opportunity 

to learn was defined as the extent to which thq teachers believed that 

their students had had an opportunity to 'learn each item on a particular 

achievement -test. Lewy concluded from his research that a relationship did 

exist between opportunity to' learn a topic and proficiency in that topic. 

In view of these findings it seems as though the variable "opportunity to 

learn" can be useful in examining both test bias and the possible effects 

of schooling on student' achievement. 

# 
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• . The term "culturally biased" has most frequently been applied to tests / 
, of mental ability and aptitude. It seems as though the term wouldjbe applied 

very differently to achievement tests.. Whereas intelligence tests car) be / 

termed "biased" to the extent that certain items do, "in fact, favor .one 

subculture group over another, achievement tests can only be biased to the 

f extent that the achievement tests do not measure what is taught in the schools 

f (i.e., to the extent that they are not content valid). In other words, if 

j J 

students have not had an opportunity to learn particular items in their school, 

• r 

those particular items are "biased" against all students in that school. 

%» • 

If "biased" and non-biased items can be identified in this manner, then 

i 

it may be possible to assess the effects of schooling. -Schooling would be 
said to have an effect on student learning to the extent that students perform 
better on a test containing items that they have had an opportunity to learn 

...(unbiased items) than on a test containing items that they have not, had an 

* * 

opportunity to learn ("biased" items). . % 

In line with the above discussion the following three null hypotheses 
were investigated in the present study: 

1. There will be no racial nor socio-economic status differences in 
ti the number of vocabulary words that students have had an 

opportunity to learn. In other words, schools of various socio- 
economic status and racial make-up will not differ in the number 
of vocabulary words they teach their students. > 

j v 

2. There will be no racial or socio-economic status -differences on 
a subtest containing words that students dicLhave/an opportunity 
to learn fn school (Subtest Y) nor on a subtest of words that the 
students did not have an opportunity to learn in school (Subtest N). 

3. There will be no difference between the mean sco/es on Subtest Y 
' and Subtest N across all students. \ 
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Procedures 

Three standardized tests were used to select a pool of items. All of 
the* items. in the pool were chosen from the Reading Vocabulary Test, 
Primary Battery, Grades 2 through 4, The- number of vocabulary words selected 
was 114. Slight modifications in the item format were made when necessary to 
insure similarity of format for all of the items, 

A list of the vocabulary words, excluding the item response .alternatives, 
was given to all first through fourth 'grade teachers in four elementary Schools 
1 in a metropolitan area in South Carolina. 

The directions which accompanied the vocabulary list were as follows: 
"Place a check in front of each word on the accompanying list that you have 
taugh? in a vocabulary lesSon or incidentally as a vocabulary word. 
'Incidentally as a vocabulary word 1 means that you took time to instruct 
the class in the meaning of a new word when they encountered it in their 
course 'readings. " 

On the basis of the results of this survey each word was classified at 
the particular grade level at which it was first taught. A word was 
classified at a particular grade level if two-thirds or more of the grade- 
level teachers indicated that they had taught the word. A table was then 
drawn up for each school indicating the grade level placement of each, word. 
'From this table, two sub-lists of words were selected. The first list contained 
the words that were taught in all the schools, in either the second or third 
grade (i =17). These were designated as the words the students had had an 
opportunity to learn (Test Y). The second list contained the words that were 
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either taught in the fourth grade in all schools or not' taught at all in 
grades one through four (i = 17). These were designated as the words that 
none of the students in any of the four .schools had had an opportunity to 
learn (Test N). Words taught in grade one were eliminated b'ecause it was " 
felt that they were much less difficult words. 

The thirty-four items were then assembled ^nto a single. test and 
administered to all of the third grade students in four schools. The 
alternatives used for each item were the. same as those used in the standardized 
test from which the item was taken. KR20 reliability estimates for the two 
subtests were .66 for test Y and .67 for Test N. ' 
Sample 

The four schools were selected because of their racial and socio-economic 
status composition. School I was a middle class, virtually all-white school.- 
School II was a middle class, racially mixed school. School III was a lower 
class, racially mixed school. School IV was a lower class, virtually all 
black school. The socio-economic status designation for each school was based 
on the median father's occupational level and the median parents' educational 
level taken from the school records. 

The sample used in the study was chosen in the following manner. All 
of the black third grade students in School II (n = 21) 4.nd all of the white 
third grade students in School III (n = 24) were selected. Twenty-five white 
third grade students were randomly selected from School I and twenty-five 
black third grade students were randomly selected from School IV. 



Results 

— ■ » 

Table 1 indicates the number of words introduced at each gra'de level 
in each of the four schools. 

< ' * — • 

. Insert Table 1 About Here 

l » 

The first grade students in the white, middle class school have been exposed 
to twice as many words as the black, middle class students, and approximately 
•five times as many words as students in the two Tower class schools. After 
this initial "head start" the number of s words introduced per year does not 
differ a great deal across the schools. However, when this head start is 
examined cumulatively the school differences in opportunity to learn appear 
large. 

In order to further examine this cumulative difference a Verbal Knowledge 
Index (VKI) was calculated for the third grade students in the four schools. 
The VKI is computed by multiplying the number of words introduced each year 
by the number of years a third grade student would have had access to that 
word and' summing up'the products. An example of tfie computation of the index 
is given in the note in Table 2. 

Insert Table 2 About Here 

If we interpret the VKI as a rough index of the students' "school-induced" 
forking vocabulary^ the results indicate. that this working vocabulary of the' 
third grade students in the middle class schools is .twice as large as that of 
the lower class third graders. Also the working vocabulary of the white third 
grade students is about one and one-fourth times as large as the black third 
graders. 



These data suggest that the original lis-t of vocabulary words Ms not 
equally content valid for the students in the four schools. The list seems 
to favor the middle class students over the lower class, and, to a lesser 
extent, the white students, over the black. .This "favoritism" indicates 
differences in schools and schooling rather than ih home background. . 

o K 

In order to investigate the last two hypotheses two subtests were 
formed. One consisted of words that were said to have been taught in 
grades two or three in all schools. The other contained words that were 
not taught in. grades one, two, or three in any of the schools. Table '3 
contains the words that comprise the two subtests. 

Insert Table 3 About Here \ , 

t „ 

An index of "school effect" was computed by subtracting each student's 
score on Test N (the non-taught words) from the score on Test Y (the taught 
words). A positive s*core on' this index means that the student scored highe/* 
on the words which he had had an opportunity to learn. A negative score means 
that the student scored higher. on the words which he had not had an opportunity 
to learn. Further ,< a zero mean score for a school indicates that the scores 
on the two subtests are the same (i.e., schooling has no effect). 

A two-way analysis of variance (Race x SES) was used to investigate 
the' second hypothesis. The results of the analysis of variance are found 
in Table 4. . 

Insert Table 4 About Here 

u 

The Race and SES main effects are not significant. School effects are 
not significantl^different fop black or white students, nor for middle and 
lower socio-economic status students. The interaction term (Race x SES) 



7 



approaches statistical significance. A graphical display of tfjis interaction 
is shown in Figure 1. 



Insert Figure 1 About Here 



In. examining Figure* 1 it appears that the school effects are virtually 
the same for the white middle and white _Jower class students.. On the other 
hand the school 'effect appears to be much more marked for the' black m.iddle 
class students than the black lower class students. The school effect is 
significantly different from zero for the black middle class students (p<.01), 
^ v and for the middle^and lower class white students (p<.05). The school 

effect is not significant for the lower class black students. 

T-frts- data tends to suggest that the black middle class students benefit 
the most from schooling in terms of their vocabulary. On the other hand, 

— black lower class students benefit very little, if any. 

Across a 1 1 students 'the schooling .effect is significantly positive. 
Table 5 shows the means, standard deviations, and correlated t-test for 
C % v all students on Test Y and Test N. - 



t Insert Table 5 About Here 



Students scored significantly higher on Test Y than on Test N. However, 

despite'this'significant difference, the mean scores for Test Y do not seem- 

\ 

. to be very\)iigh (approximately eight correct out of seventeen items). 

On th^lbasis of the above observation the teachers were questioned after 

. * - 

the results jjjf the study had teen examined. The teachers were asked to list 

• the synonym&fthey used to teach their students about each of the words in 

— — > m 

Test Y„ Th;#< actually taught synonypis were then compared with alternatives 

7 i > 

available ^for each item. The average number of synonyms that, in fact, 

ERIC f XO 
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appeared as correct choices to the items on Test Y was slightly less than 

fifty per cent. In oth^r words, even though the teachers had, in fact, 

taught the words, in many cases they had used different synonyms and/or 

explanations in class than those wta-ch appeared as item choices. 

Conclusions and Implications ' m y<B , 

It has been 'suggested that when achievement tests are*used, test bias 

be equated with content validity.. When this is done, the results of the 

study suggest that the vocabulary subtests of various standardized tests 

are biased in favor of white and middle class students. The social class 

* 

bias appears to be stronger than the racial bias. * , 

One potential problem in establishing content validity concerns the 
"correct" answer called for by the test manufacturer . In order to establish 
content validity it seems necessary to determine the answers to two separate, 
but related, questions. First, did the student have an opportunity to learrf 
this* content? Second, did the student learn the content in a manner which 
will allow him to- answer the item correctly? 

It has further been suggested that the discrepancy of the scores on a 
test containing items the students have had an opportunity to learn and a 
test containing items they have not* had an opportunity to learn be used as 
an index of school effects. The results of this study suggest that when 
this index is used, schools do have a significant effect on achievement. 
More specifically, it is concluded that schooling is most important for 4 
the development of the vocabulary of black, middle class students. Schooling 
seems to be virtually ineffective. in developing the vocabulary of black, 
lower class students. * 

11 
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Table 


1 






■ • . 1 * ■ " 
. * Number of Words 


Introduced 


at Each -Grade Levels 

1 






ur auc £ 


r **" 

Grade 3 


Grade 4 


not lauyriL 


White, Middle'- 71 


" ,7'. 


16 - 


— * — 

15 


> 

5 


Black, Middle "32 • 


. 31" 


<** 22 


•',7 * 


12 


White, Lower ' 15 . 

> . *• „ ► 


* . 15 


'34 


20 « 


30 


Black, Lower > 12 


; i3 


S . 32 \ 


23 


34 






- * i— 
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- — — — ; t^s^: — 


** 




" 
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' .Table 2 

Verbal Knowledge Index for Grade 3 Students' in Each School 



Type of School 



Verbal Knowledge Index 



Ratio of VKI (School X/ 
ISIack, Lower) 



White,, Middle • 
Black, Middle 
White, Lower 
.Black, Lower 



243' 




2.58 


180 




1?9,1 


10?' 




, 1.15 








94 ■ « 




.1.00 



♦ 'Note. - The Verbal Knowledge Index is computed by myl tidying the number of 
fjords introduced each year by the number of years a third-grade student would 
Jw^had- acqess to thdt word. For example, a third -grade student in the 
wh/fte, middle class school was introduced, to. 71 words in first grade. By the 
third grade he would have had -an opportunity to use that word for three years. 
Thus for his first grade year the verbal knowledge index is 71- x 3 or 213. 
For the second grade the VKI is 7 x.2 or'14 and. for the third grade the VKI . 
is 16 x 1 or 16. The composite index is the sum of these products, i.e., 
213 + 14 + 16 = 243. " 
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Table' 3 

t 

List of Words Taught and Non-Taught tn School 



Test Y (Taught in Grades^ 2 or 3) Test N (Not Taught in Grades 2 or 3) 



* 

* Perhaps 

Slip 
Tldeas f 

Sure 

Decide 

Village . 

Smiling ' • 

instrument 
1 Path. 

Hopped 

Carry 

Investigate 

Discover 

Bald 

Unus # ual 

Envelope 

Multiply 



4- 



Pretending 

Select 

Exami ne 

Journey ' 

Manufactured 

Precious 

Refinements 

Avenue 

Shiver 

Approach ^ 

Destroy 

Halt 

Platform 
Offer - 
Deliver 
Defeat 
Model , 



Note, Norr-Jaugfr 
earlier than Grade 4, 




3 meansthat the word was taught no 
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Source 



Race 
SES 

Race x SES 



Table 4 



Analysis of Variance of the Effect of Schooling 
on Racial and Socioeconomic Status Groups 



df 


SS 


.MS 


F-value 


1 


0.91 


0.91 


0.15 


1 


9.65 


9.65 . 


1.64 


1 


16.32 


•16.32 


2.78 



.70 
.20 
.10 



f 



16 



ERIC 



Table 5 

i 

Means, Sta-ndard Deviations and Correlated t-test for the Jotcil 
Sample on those Words that Were and Were Not Taught in School 

(n=95) 

9 

% » 

Subtest Mean Standard Deviation 1 



Taughf Words (Test Y) , 7.72 
Kon-Taught Words' (Test N) 6.82 



3.15 
3.05 



3.60 



.01 



-x7 



Figure 1. 



A Schematic Diagram of the Differential Effects 
of Schooling on Various Racial, Socio-Economic 
Status Students 4 .-v 
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